There  is  no  'A"  in  CD&E,  neither  for  Analysis  nor  for  Anarchy  -  Ensuring 
Scientific  Rigour  and  Analytical  Structure  while  maintaining  Military 

Relevance  and  Artistic  Freedom 

Mr  Thomas  Ekstrom 

FOI,  The  Swedish  Defence  Research  Agency 
SE-164  90  Stockholm 
Sweden 

thomas.ekstrom@foi.se 


ABSTRACT 

As  a  consequence  of  the  ending  of  the  Cold  War,  the  defence  forces  of  many  countries  in  the  world  are 
going  through  radical  transformation  and  reformation.  For  many  countries,  Concept  Development  and 
Experimentation  (CD&E)  has  become  the  main  vehicle  for  developing  and  testing  new  ideas  during  this 
transformation.  The  Swedish  Armed  Forces  (AF)  has  also  adopted  CD&E  as  its  main  instrument  in  its 
endeavour  to  reform  its  forces.  In  order  to  ensure  that  the  methodology’  was  properly  known  throughout 
the  Swedish  AF  in  general,  and  among  the  officers  directly  involved  in  CD&E  in  particular,  it  was 
decided  by  the  Swedish  AF  Head  Quarters  (HQ)  that  a  CD&E  Handbook  should  be  produced.  The  author 
of  this  paper  was  one  of  the  members  of  a  small  working  group  that  was  assigned  the  task  of  producing 
such  a  handbook.  In  the  extensive  literature  study  that  was  the  initial  step  in  this  work,  it  was  established 
that  most  of  that  which  had  hitherto  been  written  about  CD&E  was  either  describing  different 
classifications  and  categorisations  of  different  types  of  concepts  that  could  be  developed,  or  was  focusing 
on  the  different  types  of  experiments  which  should  ensue  to  test  the  concepts  that  were  being  developed.  In 
comparison,  relatively  little  was  written  about  other  analytical  methods  that  could  be  used  in  order  to 
ensure  that,  at  all  times,  “the  right  concept’’  was  being  developed  and  that  it  was  being  developed  in  “the 
right  way”,  i.e.  to  make  sure  that  the  maturing  concept  was  tested  throughout  its  development  in  order  to 
assure  quality,  ensure  transparency  and  in  order  to  provide  an  audit  trail.  The  purpose  of  this  paper  is  to 
present  a  case  for  why  a  structured  analytical  support  is  needed  in  order  to  ensure  that,  at  all  times,  “the 
right  concept”  is  being  developed  and  that  it  is  being  developed  in  “the  right  way”,  i.e.  to  promote  an 
analytical  approach  to  avoid  “anarchy”  in  CD&E  by  the  introduction  of  analytical  support  as  quality 
assurance.  For  managers  of  CD&E,  it  is  necessary >  to  be  able  to  state  that  scarce  resources  are  being  used 
to  develop  the  right  concept.  For  managers,  concept  developers  and  experimenters,  it  is  necessary  to  be 
able  to  state  that  the  scarce  recourses  are  being  used  to  develop  and  test  the  concept  in  the  right  way.  For 
the  analyst,  it  is  necessary  to  provide  managers,  concept  developers,  and  experimenters  with  the 
appropriate  analytical  support  to  make  sure  that  the  right  concept  is  developed  in  the  right  way,  but  also 
in  order  to  assure  quality,  ensure  transparency  and  to  provide  an  audit  trail.  For  the  scientist,  it  is 
necessary >  to  provide  the  CD&E  community  with  adequate  input,  bearing  in  mind  that  experiments  are 
being  conducted  on  social  systems,  i.e.  systems  which  to  a  large  extent  involve  human  beings  and  their 
behaviour,  not  natural  systems,  i.e.  systems  which  are  being  governed  by  the  laws  of  nature. 


1.0  INTRODUCTION 

Two  decades  ago,  the  Berlin  Wall  came  down,  East  and  West  Germany  were  reunited,  the  Soviet  Union 
(SU)  was  dissolved,  and  the  Warsaw  Pact  (WP)  broke  up,  effectively  putting  an  end  to  the  era  of  the  Cold 
War.  In  the  United  States  (US),  the  ending  of  the  Cold  War  and  the  ensuing  changes  in  defence  and 
security  policies,  in  combination  with  the  revolutionary  development  in  the  area  of  Information  and 
Communication  Technology  (ICT)  and  Lessons  Learned  (LL)  from  the  First  Gulf  War,  eventually  led  to 
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an  unprecedented  military  transformation,  labelled  a  Revolution  in  Military  Affairs  (RMA).  Initially,  the 
RMA  was  based  on  concepts  such  as  Dominant  Battle-space  Awareness  (DBA)  and  Precision  Engagement 
(PE).  Later,  new  concepts,  like  Network  Centric  Warfare  (NCW),  would  contribute  to  the  transformation. 

In  Sweden,  both  the  military  and  the  politicians  were  inspired  by  the  development  in  the  US,  and  about  a 
decade  ago,  the  Swedish  Government  decided  that  the  Swedish  defence  should  be  transformed  from  a 
Cold  War  national  defence,  designed  to  defend  Sweden,  to  a  Post  Cold  War  operational  defence,  designed 
to  be  flexible  and  deployable  throughout  the  entire  globe.  As  a  consequence  of  this  decision,  the  Swedish 
Government  assigned  the  Swedish  AF  with  the  formidable  task  of  developing  a  concept  for  a  future 
deployable  defence,  based  on  a  network  concept,  initially  labelled  a  concept  for  a  Network-Based  Defence 
(NBD).  Later,  the  network  based  aspect  of  the  new  defence  would  become  subdued  in  favour  of  new 
concepts  such  as  Effects  Based  Operations  (EBO);  the  Effects  Based  Approach  to  Operations  (EBAO);  the 
Comprehensive  Approach  (CA);  and,  most  recently,  expeditionary  operations. 

The  ensuing  development  within  the  Swedish  AF  was  divided  into  four  distinct  Command  and  Control 
(C2)  system  development  projects  in  the  areas  of  Methodology  (M),  Technology  (T),  Personnel  (P),  and 
Organisation  (O),  labelled  C2-system  M,  C2-system  T,  C2-system  P,  and  C2-system  O,  respectively.  The 
development  project  C2-system  T  was  assigned  to  the  Swedish  Defence  Materiel  Administration  (FMV), 
i.e.  the  Swedish  Defence  Procurement  Agency  (DPA),  and  got  underway  more  or  less  immediately.  The 
development  project  C2-system  M,  and  later  also  C2-system  P,  was  assigned  to  the  Swedish  Joint  Forces 
Command  (JFC).  After  a  slow  start,  these  two  projects  eventually  got  momentum  once  the  Joint  Concept 
Development  and  Experimentation  Centre  (JCD&EC)  was  established  as  the  Swedish  experimental 
platform.  The  fourth  development  project,  C2-system  O,  was  never  initiated,  since  it  was  decided  that  the 
organisational  issues  would  be  dealt  with  once  the  other  parts  of  the  new  C2-system  were  in  place. 

The  development  project  C2-system  T,  which  got  a  head  start,  decided  to  use  the  IBM  Rational  Unified 
Process  (RUP),  which  is  an  iterative  software  development  process  framework,  as  its  preferred  method  of 
development.  Once  the  development  project  C2-system  M  started  its  development,  it  was  decided  that  this 
development  project  should  have  the  same  development  method  as  the  development  project  C2-system  T, 
i.e.  RUP.  After  some  adaptation,  RUP-SE  (Systems  Engineering)  was  adopted  as  the  preferred 
development  method  by  the  project.  Over  time,  however,  many  other  development  methods  were  utilised, 
and  evolutionary  development  was  a  principle  that  was  established  early  on  in  the  development.  About 
four  years  ago,  when  the  project  was  approaching  its  deadline,  i.e.  2006-12-31,  a  reasonably  stable  Modus 
Operandi  (M.O.)  had  been  established  within  the  development  project  C2-system  M.  This  method 
consisted  of  what  can  only,  in  retrospect,  be  described  as  concept  development  followed  by  tests  in  the 
form  of  experiments.  It  was  soon  realised  that  this  M.O.  had  a  striking  resemblance  to  what  was  known 
about  a  method  that  was  becoming  increasingly  popular  in  the  international  transformational  community, 
i.e.  CD&E.  What  was  known  in  Sweden  about  the  CD&E  method  was,  however,  precious  little,  and,  for 
some  reason,  it  proved  difficult  to  identify  a  thorough  description  of  the  method  anywhere  among  the 
countries  that  were,  allegedly,  developing  concepts  in  accordance  with  the  new  method.  It  was,  however, 
surmised  that  the  two  development  methods  had  more  similarities  than  differences. 

Four  years  ago,  the  three  development  projects  C2-system  M,  C2-system  T,  and  C2-system  P  were 
terminated  [1]  in  accordance  with  the  original  plan.  However,  the  JCD&EC  lived  on,  and  the  development 
activities  were  continued  within  the  traditional  military  organisation.  When  the  development  projects  were 
terminated,  it  was  decided  by  the  Swedish  AF  HQ  [2]  that  all  concept  development  within  the  Swedish  AF 
should  be  pursued  in  accordance  with  the  development  method  that  had  been  utilised  by  the  development 
project  C2-system  M,  i.e.  a  method  that  had  never  been  described  in  any  detail,  but  which  was  considered 
to  be  identical  with,  or  at  least  very  akin  to,  the  international  CD&E  method.  At  the  JCD&EC,  the 
implications  of  the  decision  by  the  HQ  were  immediately  realised.  Within  the  Swedish  AF,  development 
centres  and  development  units  within  numerous  schools  would  soon  look  to  the  JCD&EC  for  guidance 
regarding  the  utilisation  of  the  CD&E  method.  The  Swedish  National  Defence  College  (SNDC),  which 
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very  swiftly  initiated  academic  CD&E  courses,  would  certainly  soon  look  to  the  JCD&EC  for  support. 
Within  the  JCD&EC  it  was  also  realised  that  if  “ all  concept  development  within  the  Swedish  AF ’  should 
follow  the  CD&E  method,  this  would  have  to  be  done  at  development  centres  and  development  units 
within  schools  without  the  support  of  independent  analysts  and  scientists.  In  fact,  it  would  have  to  be  done 
without  the  direct  support  of  the  JCD&EC.  It  was  promptly  decided  by  the  JCD&EC  to  produce  “The 
Swedish  Armed  Forces  CD&E  Handbook”.  It  was  also  decided  that  the  handbook  should  not  only  include 
descriptions  of  concepts,  concept  development  and  experiments,  but  also  descriptions  of  the  other 
analytical  tools  that  could  be  utilised  during  the  concept  development.  In  addition,  the  handbook  should 
also  include  the  managerial  perspective,  e.g.  the  decision  making  process. 

The  author  of  this  paper  was  one  of  the  members  of  a  very  small  working  group  that  was  assigned  the  task 
of  producing  such  a  handbook.  In  the  extensive  literature  study  that  was  an  initial  step  in  this  work,  it  was 
established  that  most  of  that  which  had  hitherto  been  written  about  CD&E  could  be  categorised  as 
providing  contributions  by:  describing  different  definitions,  classifications  and  hierarchies  of  military 
concepts  [3,  4,  5,  6,  7,  8];  generally  describing  concept  development  [3,  4,  8,  9,  10];  addressing  different 
types  of  experiments  [4,  5,  10];  or  focusing  on  how  these  experiments  should  be  planned,  executed  and 
evaluated  [10,  11,  12,  13,  14,  15,  16,  17].  Regarding  different  forms  of  analytical  tools  at  disposal,  some 
contributions  could  be  identified,  but  none  of  these  were  detailed  [3,  4,  5,  9,  10].  On  an  overall  level,  there 
were  descriptions  of  the  entire  CD&E  process  [3,  4,  5,  9,  10],  From  the  managerial  perspective,  only  one 
contribution  could  be  identified  [5].  On  a  more  detailed  level  regarding  the  analytical  methods  that  could 
be  used  in  order  to  ensure  that,  at  all  times,  “the  right  concept”  was  being  developed  and  that  it  was  being 
developed  in  “the  right  way”,  i.e.  to  make  sure  that  the  maturing  concept  was  validated  throughout  its 
development,  in  order  to  assure  quality,  ensure  transparency  and  in  order  to  provide  an  audit  trail,  precious 
little  was  found  [4].  However,  ambitious  initiatives  like  the  Integrated  Analysis  and  Experimentation 
Campaign  (IAEC)  [9,  10]  were  referred  to  and  comprehensively  described,  but  would  seem  to  presuppose 
the  participation  of  an  external,  objective  organisation,  with  the  necessary  competence  to  plan  and  execute 
the  Integrated  Analysis  and  Experimentation  (IAE)  activities. 

In  this  paper,  some  of  the  theoretical  findings  of  the  literature  study  are  presented  in  Chapter  2.0.  In 
Chapter  3.0,  five  different  perspectives  on  CD&E,  primarily  empirically  based  on  the  author’s  experience 
of  CD&E  in  practise,  are  presented.  In  Chapter  4.0,  these  different  perspectives  are  used  in  order  to 
discuss  why  a  analytical  structure  is  required  in  CD&E.  The  conclusions  of  the  paper  are  presented  in 
Chapter  5.0.  Finally,  acronyms  and  references  are  presented  in  Chapter  6.0  and  7.0,  respectively. 

The  personal  reflections  reported  in  this  paper  are  based  on  participation  in  several  projects  that  were 
commissioned  to  FOI,  the  Swedish  Defence  Research  Agency  (DRA),  by  the  Swedish  AF.  The  Swedish 
AF  has  not,  however,  explicitly  commissioned  any  research  leading  to  the  preparation  of  this  paper.  The 
opinions,  conclusions,  and  recommendations  expressed  or  implied  within  this  paper  are  consequently 
solely  those  of  the  author.  They  do  not  necessarily  represent  the  views  of  the  Swedish  Government,  the 
Swedish  Ministry  of  Defence  (MoD),  the  Swedish  AF,  FOI,  or  any  other  Swedish  Government  agency. 
The  paper  has  been  cleared  for  public  release  and  for  unlimited  distribution  by  FOI. 


2.0  CONCEPT  DEVELOPMENT  AND  EXPERIMENTATION  IN  THEORY 

Once  the  project  to  produce  a  Swedish  CD&E  Handbook  commenced,  it  was  quickly  established  that 
nowhere  had  an  identical  project  already  been  done.  Similar  projects  were  identified  [9,  10],  but  none  with 
the  same  level  of  ambition,  i.e.  to  produce  a  document  that  could  be  used  for  education  and  as  a  theoretical 
guide  for  those  managers,  concept  developers,  experimenters,  analysts,  etc.  that  should  use  the  CD&E 
method  in  practise.  As  far  as  the  small  working  group  could  find  out,  there  was  no  CD&E  Handbook 
already  in  existence  anywhere  in  the  world.  In  fact,  there  were  very  few  comprehensive  descriptions  of  the 
CD&E  method  to  be  found  at  all  [4,  9,  10].  As  for  the  explicit  components  of  CD&E  method,  i.e. 
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descriptions  of  military  concepts,  military  concept  development,  and  military  experimentation,  the 
situation  was  decidedly  better,  even  if  consensus  regarding  terminology  and  definitions  was  non-existent. 
From  an  analytical  point  of  view,  however,  existing  documentation  left  the  analyst  wanting  regarding  the 
more  implicit  components  of  the  CD&E  method.  Descriptions  of  how  analysis  and  analytical  methods  and 
tools  should  be  used  in  conjunction  with  the  CD&E-method  were  few  in  existence,  and  without  detail. 

During  the  first  half  of  2008,  the  author  of  this  paper  performed  an  extensive  literature  study,  primarily  in 
order  to  answer  six  questions:  “What  is  a  military  concept?”;  “What  is  military  concept  development?”; 
“What  is  military  experimentation?”;  “What  is  military  concept  development  and  experimentation?”; 
“What  is  validation  of  a  military  concept?”;  and  “What  is  analysis  in  military  concept  development  and 
experimentation?”.  In  this  chapter,  some  of  the  identified  answers  to  these  questions  are  presented  as  a 
background  to  the  ensuing  discussion.  When  the  literature  study  commenced,  it  was  already  appreciated 
that  it  would  not  be  trivial,  if  at  all  possible,  to  identify  a  universally  accepted  definition  regarding  what 
CD&E  was.  It  was  not  anticipated,  however,  that  there  should  also  be  such  a  widespread  lack  of  consensus 
regarding  what  a  military  concept,  military  concept  development,  and  military  experimentation  is. 

There  are  many  definitions  of  what  a  concept  is  in  the  military  context.  According  to  NATO  AAP-6  a 
concept  is  defined  as  “A  notion  or  statement  of  an  idea ,  expressing  how  something  might  be  done  or 
accomplished,  that  may  lead  to  an  accepted  procedure ”  [6].  The  Canadian  Forces  Experimentation  Centre 
(CFEC)  defines  a  concept  as  “ The  identification  of  a  problem  area  or  concern  together  with  a  hypothesis 
to  remedy  the  situation  or  solve  the  problem.  It  may  also  be  revolutionary’,  evolutionary’  and/or  innovative 
forces  that  drive  the  requirement  for  new  technology >,  new  structure,  new  business  processes  and  partners ” 
[8].  Yet  another  contribution  states  that  “A  military  concept  is  the  description  of  a  method  or  scheme  for 
employing  specified  military  capabilities  in  the  achievement  of  a  stated  objective  or  aim.  This  description 
may  range  from  broad  to  narrow.  It  may  range  from  describing  the  employment  of  military  forces  in  the 
broadest  terms  and  at  the  highest  levels  to  specifying  the  employment  of  a  particular  technology >  system  or 
the  application  of  a  particular  training  system ”  [3]. 

In  addition  to  the  different  definitions,  there  are  several  possible  ways  of  categorising  different  types  of 
concepts.  The  context  is,  of  course,  of  paramount  importance.  In  this  paper  only  military  concepts  are 
considered.  If  the  temporal  aspect  is  of  interest,  concepts  can  be  divided  into  historical,  current,  and  future 
concepts  [3].  There  are  four  levels  of  military  concepts,  constituting  a  hierarchy:  institutional  concepts, 
which  describe  military  institutions;  operating  concepts  (strategic,  operational,  and  tactical  concepts), 
which  describe  how  military  forces  operate;  functional  concepts,  which  describe  the  performance  of 
individual  military  functions  or  sub-functions;  and  enabling  concepts,  which  describe  the  capabilities 
required  in  order  to  perform  military  functions  or  sub-functions  [3].  If  the  ground  for  classification  is  the 
purpose  of  the  concept,  concepts  can  be  divided  into  capstone,  enabling,  and  applied  concepts  [4], 
Concepts  can  also  be  classified  based  on  the  maturity  of  the  concept,  whereby  concepts  can  be  defined  as 
analytical,  interim,  and  applied  concepts  [5].  There  are,  of  course,  other  types  of  military  concepts  with 
which  the  concept  in  CD&E  should  not  be  confused.  Three  of  these  other  types  of  concepts  are  Concept  of 
Operations  (CONOPS)  [6],  Concept  of  Employment  (CONEMP)  [5],  and  Concept  of  Use  (CONUSE)  [5]. 

The  Technical  Cooperation  Program  (TTCP)  made  a  comparison  [7]  among  its  member  nations  (Australia, 
New  Zealand,  US,  UK  and  Canada)  regarding  their  utilisation  of  different  concept  terminologies  and 
hierarchies.  The  TTCP  also  made  an  attempt  at  establishing  the  smallest  common  denominators.  The 
result  is  illustrated  in  Table  1.  Terms  marked  with  an  asterisk  (*)  in  the  table  are  not  formally  parts  of  that 
country’s  concept  hierarchy,  but  represent  attempts  to  identify  corresponding  activities  in  the  hierarchical 
levels  where  these  countries  would  otherwise  have  had  gaps  in  the  table. 

As  is  demonstrated  by  Table  1,  there  is  no  consensus  concerning  concept  terminology  and  concept 
hierarchy  among  the  participating  countries  in  the  TTCP.  In  fact,  there  is  not  even  any  consensus 
regarding  the  number  of  levels  in  the  hierarchy.  The  column  to  the  far  right  in  Table  1  represents  the 
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TTCP  attempt  at  finding  the  common  denominator  among  terminologies  and  hierarchies.  Further,  it 
deserves  to  be  mentioned  that  while  the  US  and  Canada  advocate  a  hierarchical,  top-down  perspective,  the 
UK  propagate  an  approach  based  on  processes,  and  Australia  emphasises  the  importance  of  relationships. 


UK 

Australien 

Kanada 

USA 

TTCP 

Analytical  Joint 

Joint 

Capstone 

Capstone  Concept 
for  Joint  Operations 
(CCJO) 

Defence  level 

*Military  Task 

Operating 

Concept 

Australian 

Military  Strategy 

Joint  Operating 

Concepts  (JOC) 

Mission 

Analytical 

Environmental 

Environmental 

Environmental 

*  Service  Concepts 

Environmental 

*Hybrid  Concepts 

Integrating 

Integrating 

*  Joint  Concepts 

Integrating 

Interim  /  Applied 

Enabling 

Functional 

Joint  Functional 
Concepts 

Functional  and 

Enabling 

Interim  /  Applied 
Sub-concepts 

Joint  Integrating 

ConEmp 

*ConOps/ 

ConEmp 

*ConOps 

*  Capability 

Development 

Document 

Capability 
Requirements  and 
Acquisition 

ConUse 

*ConOps 

*Capability 

Production 

Document 

Table  1:  Anglo-Saxon  concept  terminology  and  concept  hierarchies  [6], 


Definitions  of  what  concept  development  is  are  not  as  abundant  as  are  those  of  what  a  concept  is.  The 
Canadian  Forces  Experimentation  Centre  (CFEC)  defines  concept  development  as  “ The  process  by  which 
ideas  are  refined,  accepted  or  rejected  through  the  postulation  of  increasingly  detailed  hypothesis 
statements.  This  process  permits  the  exploration  of  any  concept  to  determine  its  merit  and  feasibility' ’  [8]. 
This  definition  eloquently  and  explicitly  summarise  those  fundamental  elements  of  concept  development 
around  which,  perhaps,  a  consensus  regarding  the  process  of  concept  development  does  exist.  Hence, 
concept  development  can  be  considered  to  be  a  process  by  which  a  concept  matures  by  successive  testing. 

Concepts  are  not  initiated  in  full  form,  nor  are  they  fully  realised  after  a  few  iterations.  Concept 
development  is  not  an  orderly,  sequential  process,  resulting  in  an  engineered  solution,  where  the  final 
result  is  fully  blueprinted  at  the  beginning  of  the  process.  Instead,  military  concepts  tend  to  form 
iteratively  and  incrementally  over  time,  and  military  concept  development  is  a  process  of  exploration  and 
experimentation  and  tends  to  unfold  as  a  hypothesis-antithesis-synthesis  dialogue  [3].  Figure  1  illustrates 
the  iterative  nature  of  concept  development,  as  well  as  the  successive  testing  of  the  maturing  concept. 

Military  experimentation  is  the  main  topic  of  several  publications  [9,  10,  11,  12,  13,  14,  15,  16,  17],  the 
contents  of  which  will  not  be  repeated  here.  There  are,  however,  several  ways  of  categorising  experiments, 
which  deserves  mentioning.  Depending  on  the  purpose;  which  can  be  to  explore  the  effectiveness  of 
something  that  has  yet  to  be  tested,  to  investigate  the  validity  of  a  hypothesis,  or  to  demonstrate  a  known 
truth,  of  the  experiment;  it  can  be  classified  as  discovery,  hypothesis-testing,  or  demonstration.  Depending 
on  the  scope  of  the  experiment,  i.e.  if  the  entire  concept,  parts  of  the  concept,  or  only  one  part  of  the 
concept,  is  being  tested,  the  experiment  can  be  classified  as  Main  Event  (ME),  Minor  Integrating  Event 
(MIE),  and  Limited  Objective  Experiment,  (LOE),  respectively.  Most  experiment  employs  some  form  of 
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simulation.  Generally  speaking,  simulations  can  be  divided  into  different  categories  based  on  a  number  of 
different  criteria.  Examples  of  such  criteria  include  the  handling  of  time  (static  or  dynamic  simulation),  of 
probability  (deterministic  or  stochastic  simulation),  and  of  variables  (continuous  or  discrete  simulation);  as 
well  as  of  how  the  simulation  is  finished  (terminating  or  non-terminating  simulation).  In  the  military 
domain,  simulations  are  traditionally  categorised  as  constructive  simulation,  analytic  war  games,  virtual 
simulation,  and  live  simulation  (field  experiments). 


The  combination  of  concept  development  and  experimentation,  i.e.  CD&E,  is  described  by  NATO  Allied 
Command  Transformation  (ACT)  as  a  “ Companion  process  to  the  Capability  Development  Process 
(CDP)  ...  provides  important  means  to  identify  possible  solutions  to  capability  gaps  ...  CD&E  is  a  driving 
tool  for  NATO’s  transformation  ...  enables  structured  development  of  creative  and  innovative  ideas  ... 
provides  mechanisms  to  turn  these  ideas  into  viable  solutions  ...  provides  a  pathway  for  new  idea 
development  ...  provides  focus  for  future  capability  development  ...  promotes  collaboration  amongst  the 
Allies ”  [4].  The  Bundesamt  fur  Wehrtechnik  und  Beschaffnung  contributes  with  the  following  definition 
of  CD&E:  “ designates  a  method  which  allows  us  to  predict,  by  way  of  experimentation,  whether  certain 
concepts,  theoretical  constructs,  sub-systems  or  systems  are  apt  to  meet  the  requirements  imposed  by  the 
transformation  process  and  can  be  constructively  integrated  into  an  overarching  system”  [18]. 

An  essential  part  of  CD&E  is  validation  of  the  developing  concept.  The  interdependence  between  concept 
development  and  the  maturing  concept  has  been  beautifully  described  as  “ A  successful  concept  must 
undergo  a  validation  process  by  which  it  is  tested  and  eventually  accepted  or  rejected  by  the  institution. 
Any  important  military  concept  under  consideration  should  be  the  subject  of  an  open  and  honest  debate 
within  the  institution.  The  validation  process  provides  a  sort  of  crucible  through  which  the  concept  must 
pass  -  strengthening  the  concept  in  the  process  if  it  survives.  In  this  way  validation  and  development  are 
closely  linked.  Like  development,  validation  often  tends  to  be  disorderly.  It  occurs  both  formally  and 
informally.  Both  are  necessary >  in  validating  a  concept.  A  concept  may  have  been  officially  approved,  but 
is  not  truly  validated  until  it  has  been  accepted  by  the  stakeholders  of  the  institution.  Formal  validation 
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takes  place  through  workshops,  war  games,  experiments  or  other  activities  held  by  proper  authority  for 
the  express  purpose  of  evaluating  the  concept.  Informal  validation  occurs  in  the  operating  forces, 
professional  schools  and  the  institution  at  large  in  the  form  of  field  and  map  exercises  and  articles  in 
professional  journals.  All  are  important  in  validating  the  concept.  Acceptance  takes  some  unavoidable 
period  to  grow.  Some  elements  of  a  concept  may  gain  acceptance  more  quickly  than  others.  Some 
elements  may  gain  acceptance  while  others  are  rejected.  Just  as  it  is  in  the  development  process,  strong 
leadership  is  essential  during  the  validation  process.  As  a  concept  gains  acceptance  over  time  it 
transforms  from  a  tentative  hypothesis  to  a  more-or-less  accepted  conclusion.  Reflecting  this,  the 
language  of  the  concept  document  may  become  increasingly  assertive  over  the  various  iterations.  By  the 
time  a  concept  is  formally  validated,  the  institution  should  be  comfortable  with  it  or  should  at  least  have 
had  adequate  time  to  weigh  in  on  the  subject ”  [3]. 

It  is  obvious  that  a  maturing  concept  must  go  through  an  iterative  process  during  which  is  tested  and 
validated  before  in  continues  its  development.  What  is  less  obvious  are  the  details  regarding  how  this 
validation  should  be  done,  particularly  in  the  early  iterations  of  the  process.  When  the  concept  approaches 
full  maturity,  full  scale  experiments  are  clearly  appropriate,  but  what  analytical  tools  that  should  be  used 
early  on  in  the  process  are  less  self-evident.  The  contents  of  such  a  process  is  sketchily  outlined  in  several 
publications,  e.g.  “ The  initial  exploration  of  ideas  in  military >  concept  development  may  typically  involve 
workshops,  conferences  and  articles  in  professional  journals,  and  is  typically  followed  by  seminars  and 
manual  war  games  and  only  later  by  larger-scale  exercises  and  simulations ”  [3];  “The  potential 
Integrated  Analysis  and  Experimentation  (IAE)  activities  to  support  Concept  Development  can  be  broadly 
grouped:  anticipating  likely  scientific  and  technological  advances  (and  their  implementation) ;  historical 
analysis  and  historical  research ;  workshops  and  problem  solving  techniques  (including  brainstorming) ; 
seminars;  scenario-based  workshops;  modelling  (including  simulation);  and  experimentation  (including 
Warfighting  Experimentation  (WEE)).  The  applicability  of  activities  will  vary  at  the  various  stages  of 
conceptual  development  (analytical,  interim,  applied).  Activities  with  a  broader  analysis  base  will  have 
more  utility  earlier  in  the  process,  whereas  those  with  greater  depth  will  be  more  useful  in  later  stages’’'’ 
[5];  and  “In  all  likelihood,  seminars,  workshops,  historical  analysis,  and  the  like,  will  also  be  required  as 
part  of  the  campaign  to  support  and  help  inform  the  experimenters  who  will  ultimately  address  the  overall 
question.  The  campaign  plan  process  must  take  these  other  activities  into  account  within  its  design  phase. 
The  ultimate  aim  is  to  synthesise  the  outputs  from  all  activities  into  coherent  advice  to  the  decision 
makers ”  [10].  While  analytical  components  are  explicitly  mentioned,  there  are  no  details  of  the  analytical 
methodology,  i.e.  the  “what’s,  when’s,  where’s,  how’s,  who’s,  and  why’s”  are  not  explicitly  explained. 
Evidently,  it  is  presupposed  that  an  external,  objective  organisation  with  the  necessary  competencies 
should  assist  the  concept  developers  with  planning,  execution,  and  evaluation  of  the  analytical  support. 


3.0  CONCEPT  DEVELOPMENT  AND  EXPERIMENTATION  IN  PRACTISE 

From  the  point  of  view  of  CD&E  practise  in  Sweden,  at  least  five  different  perspectives,  related  to  five 
archetypal  stakeholders,  can  be  identified  when  discussing  CD&E,  i.e.  the  perspectives  of:  the  concept 
developer;  the  experimenter;  the  manager;  the  analyst;  and  the  scientist.  In  this  chapter,  the  first  four  of 
these  perspectives  are  noticeably  and  intentionally  exaggerated,  whereas  the  remaining  one,  the  scientific 
perspective,  is  intentionally  fragmented  and  claimed  to  be  ambiguous  and  multiple,  in  order  to 
demonstrate  two  irrefutable  points.  The  first  point  being  that  depending  on  the  perspective,  the 
expectations  on  and  requirements  of  CD&E  differ  quite  significantly.  The  second  point  is  that  there  is  not 
one,  but  several,  scientific  perspectives,  of  which  that  of  the  natural  scientist  is  not  necessarily  the  most 
appropriate  for  CD&E.  The  CD&E  method  must,  within  reason,  strive  to  satisfy  the  different  stakeholders 
with  their  different  perspectives,  once  the  scientific  perspective  has  been  agreed  upon  within  the 
community.  The  author  of  this  paper  is  not  so  presumptuous  as  to  pretend  to  have  first  hand  experience 
from  all  perspectives,  but  asserts  to  have  enough  experience  in  order  to  outline  these  deliberately 
exaggerated  perspectives,  to  some  extent  inspired  by  empery,  and  to  multiply  the  scientific  perspective, 
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exclusively  encouraged  by  theory.  This  assertion  is  based  on  16  years  of  experience  in  Operational 
Research  (OR),  whereof  six  years  working  as  a  CD&E  analyst,  which  included  working  in  close 
cooperation  with  concept  developers,  experimenters,  analysts  and  managers  of  the  experimental  platform, 
and  four  years  of  part  time  studies  as  a  PhD  candidate  in  parallel,  which  have  presented  abundant 
opportunities  to  contemplate  the  different  aspects  of  the  philosophy  of  science  and  research  methodology. 

3.1  The  exaggerated  perspective  of  the  concept  developer  -  Artistic  freedom 

The  concept  developer  would  like  to  identify  a  problem  or  an  opportunity  by  himself.  He  would  then  like 
to  decide  for  himself  whether  or  not  to  develop  a  concept  based  on  his  findings.  Should  he  decide  to 
develop  a  concept  to  solve  a  problem  or  capitalise  on  an  opportunity,  he  would  like  to  have  the  resources 
he  needed,  when  he  needed  them,  and  for  as  long  as  he  needed  them.  He  would  also  like  to  decide  for 
himself  if  he  should  deliver  anything  at  all,  what  to  deliver,  when  to  deliver  it,  where  to  deliver  it,  how  to 
deliver  it,  and  to  whom  to  deliver  it.  The  concept  developer  does  not  really  want  a  CD&E  method,  a 
CD&E  process  or  any  other  form  of  CD&E  structure.  He  wants  his  artistic  freedom  to  do  what  he  wants 
to,  when  he  wants  to,  where  he  wants  to,  how  he  wants  to,  and  with  whom  he  should  do  it. 

The  concept  developer  does  not  really  see  the  point  in  testing  the  maturing  concept  throughout  its 
development.  The  concept  developer  is  not  interested  in  providing  the  manager  with  information  at  regular 
intervals  concerning  the  current  status  of  the  maturing  concept.  The  concept  developer  does  not  want  to 
explore  different  alternative  solutions,  regardless  of  what  the  analyst  advocates.  The  concept  developer  is 
not  interested  in  serving  the  experimenter  with  his  required  input  several  months  in  advance  of  any 
impending  experiments  that  still  loom  in  the  far  distant.  How  should  he,  he  asks  himself,  months  in 
advance  to  boot,  know  which  questions  that  he  may  want  a  distant  experiment  to  address? 

3.2  The  exaggerated  perspective  of  the  experimenter  -  Timely  order 

The  experimenter  knows  one  thing  for  certain;  the  Experiment,  with  a  “capital  E”,  was  scheduled  in  time 
more  than  one  year  ago,  in  order  to  make  sure  that  the  necessary  resources  could  also  be  allocated  and 
scheduled  in  due  course.  Hence,  the  necessary  resources,  i.e.  the  experimental  platform  itself,  as  well  as 
the  required  personnel,  etc.  have  been  allocated  and  scheduled  to  participate  in  the  Experiment  for  at  least 
one  year  by  now.  The  experiments  itself  is  less  than  six  months  away.  While  the  resources  are  in  place, 
even  if  the  experimenter  has  learned  the  hard  way  that  some  of  his  resources  will  not  show  up  for  the 
Experiment,  he  misses  one  very  important  input  in  order  to  perfect  the  definite  plan  for  the  Experiment; 
which  questions  does  the  concept  developer  want  to  have  answered  through  the  Experiment? 

The  experimenter  does  nor  really  care  what  questions  the  concept  developer  wants  to  have  answered.  The 
experimenter  does  not  even  care  which  answers  the  Experiment  will  provide  the  concept  developer  with. 
The  only  thing  the  experimenter  cares  about  is  getting  the  correct  input  at  the  correct  point  in  time,  so  that 
he  can  produce  the  final  plan  for  the  experiment,  and  devote  his  attention  to  actually  receiving  the 
necessary  resources  on  the  correct  day,  so  that  the  Experiment  will,  in  fact,  be  able  to  answer  any 
questions  at  all.  The  experimenter  does  not  really  care  about  a  CD&E  method,  a  CD&E  process  or  any 
other  form  of  CD&E  structure.  He  recognises,  however,  that  a  method,  process,  or  other  form  of  structure 
may  assist  him  in  getting  the  necessary  input  in  time.  How  should  he,  he  asks  himself,  otherwise  make 
sure  that  the  concept  developer  actually  provides  him  with  the  necessary  input  in  time? 

3.3  The  exaggerated  perspective  of  the  manager  -  Military  relevance 

The  manager  of  the  experimental  platform,  who  is  also  the  superior  officer  of  the  concept  developer  and 
the  experimenter,  has  a  perspective  that  is  at  least  twofold.  First  of  all,  he  knows  that  his  superior  officer, 
i.e.  the  HQ,  will  at  any  given  point  in  time  come  to  him  with  the  request  to  put  another  concept  into  his 
“concept  development  and  experimentation  factory”.  Regardless  of  whether  or  not  the  requirement  from 
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the  HQ  is  a  whim,  based  on  some  new  three  letter  acronym,  he  knows  that  he  must  be  able  to  answer 
questions  regarding  the  status  of  the  ongoing  concept  development  projects,  including  predictions  of  the 
future  success,  as  well  as  questions  regarding  consequences  for  the  ongoing  development,  should  the  HQ 
decide  to  push  a  new  concept  into  the  factory.  In  order  to  give  the  HQ  relevant  answers  at  any  given  point 
in  time,  he  knows  that  his  factory,  i.e.  the  experimental  platform,  must  have  a  method,  process  or  other 
form  of  structure,  which,  over  time,  can  provide  him  with  adequate  information  regarding  the  status  of  the 
ongoing  concept  development  projects.  The  yearly  Experiments  will  not  be  sufficient  to  provide  him  with 
the  necessary  information.  He  knows  that  the  steering  group  in  the  HQ  convenes  regularly,  perhaps  on  a 
monthly  basis.  Hence  he  knows  that  he  needs  information  that  is,  preferably,  updated  at  least  once  a 
month.  Regardless  of  the  unpredictable  fads  of  the  HQ,  the  manager  also  has  an  interest  to  ensure  that  his 
scarce  resources  are  utilised  efficiently.  Hence,  secondly,  he  knows  that  he  must  provide  his  subordinates 
with  a  CD&E  method,  a  CD&E  process  or  other  form  of  CD&E  structure,  so  that  he  can  make  sure  that 
“concepts  are  being  produced  correctly”,  or,  in  other  words,  that  the  scarce  resources  at  his  disposal  are 
being  used  to  develop  the  concepts  in  a  fashion  that  is  the  most  effective. 

The  orders  from  the  HQ  are,  of  course,  not  sudden  impulses.  The  HQ  has  the  ultimate  responsibility  that, 
at  any  given  point  in  time,  the  factory  is  producing  that  which  is  most  relevant  for  the  AF.  In  order  to 
ensure  this,  the  HQ  must,  of  course,  receive  relevant  information  from  its  subordinate,  i.e.  the  manager  of 
the  experimental  platform,  so  that  it  is  possible  for  the  HQ  to  make  informed  decisions,  i.e.  to  make  sure 
that  the  “correct  concepts  are  being  developed”,  or,  in  other  words,  that  the  scarce  resources  are  being 
used  to  develop  the  concepts  that  are  most  relevant  to  the  requirements  of  the  AF.  In  short,  the  HQ 
recognises  the  need  for  gates  in  the  process,  so  that  concepts  can  be  allowed  to  enter  and  exit  the  factory, 
as  well  as  be  terminated  prematurely,  etc.,  in  an  orderly  fashion. 

3.4  The  exaggerated  perspective  of  the  analyst  -  Analytical  structure 

The  analyst  does  not  really  care  which  concepts  that  are  being  developed.  Even  if  he  has  an  interest,  it  is 
not  his  job  to  do  so.  The  military  relevance  is  a  matter  that  he  gladly  leaves  with  his  military  colleagues. 
The  analyst  cares  about  analytical  structure,  i.e.  that  concepts  are  being  developed  in  the  correct  fashion. 
The  analyst  is  convinced  that  there  has  to  be  an  established  CD&E  method,  a  CD&E  process  or  another 
form  of  CD&E  structure,  in  order  to  make  sure  that  concepts  are  being  developed  in  the  correct  fashion. 
The  analyst  gets  apprehensive  when  he  cannot  detect  even  the  slightest  evidence  to  suggest  that  any 
alternatives  are  being  investigated  by  the  concept  developers.  The  analyst  tries  to  argue  that  it  is 
impossible  for  concept  development  to  be  that  linear.  Issues  dealt  with  by  the  JCD&EC  ought  to  be 
complex  enough,  the  analyst  argues,  that  different  alternatives  must  be  pursued  in  order  to  develop  the 
concept  in  a  correct  fashion.  The  analyst  argues  that  a  trial  and  error  approach  ought  to  be  the  natural 
approach  in  CD&E.  The  analyst  argues  that  the  CD&E  method  must  be  iterative,  and  include  tools  that 
can  eliminate  irrelevant  development  branches  as  soon  as  possible.  The  analyst  argues  that  an  iterative 
approach,  where  the  concept  developer  uses  more  and  more  advanced  means  of  testing  the  maturing 
concept  should  be  utilised.  The  analyst  argues  that  the  advantages  of  such  an  approach  are  threefold:  it 
tests  the  maturing  concept;  it  provides  the  decision  makers  with  relevant  information  at  regular  intervals; 
and,  if  implemented  correctly,  it  also  provides  the  concept  developers  with  an  opportunity  to  create 
support  among  colleagues  outside  the  CD&E  community  through  the  dialogue  that  tests  like  seminars, 
workshops,  and  war-gaming  can  enable. 

The  analyst  is  adamant  about  the  necessity  of  a  CD&E  method,  a  CD&E  process  or  another  form  of 
CD&E  structure.  The  analyst  is  convinced  that  only  through  such  a  structured  approach  can  the  concept  be 
developed  with  the  necessary  quality  assurance;  the  necessary  information  to  decision  makers  be  provided; 
transparency  of  process  be  guaranteed;  and  an  audit  trail  be  provided.  To  realise  these  aspects  of  analytical 
structure  it  is,  of  course,  imperative  that  necessary  documentation  is  produced  throughout  the  concept 
development  process.  Hence,  the  CD&E  method,  CD&E  process  or  other  form  of  CD&E  structure  that  is 
implemented  must,  the  analyst  emphasises,  also  dictate  which  documents  that  should  be  produced,  what 


RTO-MP-SAS-081 


23-9 


There  is  no  A'  in  CD&E,  neither  for  Analysis  nor 

for  Anarchy  -  Ensuring  Scientific  Rigour  and  Analytical 

Structure  while  Maintaining  Military  Relevance  and  Artistic  Freedom 


ORGANIZATION 


the  documents  should  encompass,  when  they  should  be  produced,  how  they  should  be  produced,  by  whom 
the  should  be  produced,  etc. 

3.5  The  ambiguous  perspective  of  the  scientist  -  What  is  scientific  rigour? 

Judging  by  the  statements  of  some  commentators  within  the  CD&E  community,  it  would  seem  that  they 
champion  the  notion  that  there  is  only  one  scientific  perspective,  advocating  only  one  scientific  method, 
and  defending  only  one  idea  regarding  what  scientific  rigour  is.  In  reality  this  is,  of  course,  far  from  the 
truth.  In  reality,  there  exist  an  almost  infinite  number  of  different  possible  scientific  perspectives,  where 
the  dominating  positivist  perspective  of  the  natural  sciences  only  constitutes  one  extreme  of  an  entire, 
continuous  spectrum  of  different  perspectives  in  the  social  sciences.  There  is  no  consensus  regarding  the 
terminology  of  what  constitutes  the  other  extreme  of  the  spectrum,  but  terms  like  anti-positivism, 
interpretivism  and  phenomenology  are  frequently  used.  Somewhere  in  the  middle  of  the  road  it  is  often 
argued  that  realism,  or  scientific  realism,  has  its  justification.  Pragmatism  is  a  term  used  to  encompass 
those  scientists  who  argue  that  you  can  select  research  paradigm,  and  that  the  research  question  should 
decide  which  paradigm  you  should  choose.  Pragmatists  are  advocates  of  mixed  methodology,  or  multi¬ 
methodology,  i.e.  a  combination  of  both  qualitative  and  quantitative  research  methods.  Even  within  the 
positivist  paradigm,  where  the  scientific  method  approximately  can  be  outlined  as  consisting  of  asking  a 
question;  doing  background  research;  constructing  a  hypothesis;  testing  the  hypothesis  with  an 
experiment;  and  reporting  the  results;  consensus  is  lacking  regarding  the  testing  of  the  hypothesis. 
Verificationists  (like  Lakatos)  will  claim  that  hypothesised  regularities  can  be  verified  by  an  adequate 
experimental  research  programme,  while  falsificationists  (like  Popper)  will  maintain  that  hypotheses  can 
only  be  falsified  by  experiments  and  never  be  demonstrated  to  be  “true”. 
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Figure  2:  Framework  for  theory  building  [20]. 


A  paradigm  (as  suggested  by  Kuhn)  is  a  construct  that  specifies  a  general  set  of  philosophical 
assumptions.  These  assumptions  cover,  e.g.,  ontology,  epistemology,  axiology,  and  methodology.  One 
attempt  [19],  which  substitutes  axiology  for  view  of  human  nature,  at  describing  the  extremes  of  the 
spectrum  of  possible  paradigms  for  the  social  sciences,  labels  them  the  subjectivist  and  objectivist 
approach  to  social  science  respectively,  where  the  latter  approach  corresponds  to  the  positivistic  paradigm 


23-10 


RTO-MP-SAS-081 


There  is  no  A'  in  CD&E,  neither  for  Analysis  nor 
for  Anarchy  -  Ensuring  Scientific  Rigour  and  Analytical 
Structure  while  Maintaining  Military  Relevance  and  Artistic  Freedom 


of  the  natural  sciences.  The  objective  approach  is  characterised  as  consisting  of  a  realistic  ontology,  a 
positivistic  epistemology,  a  deterministic  view  of  human  nature,  and  a  nomothetic  methodology.  The 
subjective  approach  is  characterised  as  consisting  of  a  nominalistic  ontology,  an  anti-positivistic 
epistemology,  a  voluntaristic  view  of  human  nature,  and  an  ideographic  methodology. 

Given  the  brief  account  above,  it  should  be  obvious  that  there  is  more  than  one  scientific  perspective  in 
existence.  From  the  objectivistic  (positivistic)  point  of  view  it  would  be  argued  that  the  social  world  exists 
independent  of  our  appreciation  of  it;  that  what  happens  in  the  social  world  can  be  explained  and  predicted 
by  searching  for  regularities  and  causal  relationships  between  its  constituent  elements;  that  the  traditional 
approaches  which  dominate  the  natural  sciences  are  applicable  in  the  social  world;  that  man  and  his 
activities  should  be  regarded  as  being  completely  determined  by  the  situation  or  “environment”  in  which 
he  is  located;  and  that  research  should  be  based  on  systematic  protocol  and  technique,  i.e.  scientific  rigour 
(as  defined  by  the  positivists  of  the  natural  sciences). 
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Figure  3:  Constituent  components  of  research  quality  and  their  context  [22]. 


On  the  other  extreme  of  the  spectrum,  i.e.  from  the  subjectivistic  (anti-positivistic)  point  of  view,  it  would 
be  argued  that  the  social  world  does  not  exist  independent  of  our  appreciation  of  it;  that  the  world  is 
essentially  relativistic  and  can  only  be  understood  from  the  point  of  view  of  the  individuals  who  are 
directly  involved  in  the  activities  which  are  to  be  studied;  that  man  is  completely  autonomous  and  free- 
willed;  and  that  one  can  only  understand  the  social  world  by  obtaining  first-hand  knowledge  of  the  subject 
under  investigation,  i.e.  by  getting  close  to  the  subject  and  exploring  its  detailed  background  and  history. 

Between  these  two  extremes,  a  wide  variety  of  different  positions,  or  paradigms,  are  possible.  As  indicated 
previously,  scientific  realism  is  one  of  these.  Both  extremes  limit  their  respective  freedom  of  action. 
Flence,  from  the  perspective  of  theory  building,  the  positivist  is  limited  to  using  the  deductive,  or  the 
hypothetic-deductive  research  approach,  whereas  the  interpretivist  is  limited  to  using  the  inductive 
research  approach.  The  scientific  realist,  however,  reserves  himself  the  privilege  of  selecting  the  inductive, 
deductive,  or  the  abductive  approach  to  research,  as  he  deems  appropriate  in  each  individual  case.  Figure  1 
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illustrates  the  different  freedoms  of  action  for  the  interpretivist,  the  scientific  realist,  and  the  positivist. 

The  research  approach  at  disposal  is  not  the  only  aspect  of  the  different  paradigms  that  differentiate  them 
from  each  other.  One  very  poignant  difference  is  the  view  on  scientific  rigour.  From  the  positivistic  point 
of  view,  scientific  rigour  has  traditionally  been  discussed  in  terms  of  external  validity,  reliability,  construct 
validity,  and  internal  validity.  Many  positivists,  who  are  predominantly  quantitatively  oriented  in  their 
approach  to  science,  even  go  as  far  as  considering  their  approach  to  scientific  rigour  to  be  the  one  and 
only,  and  maintain  that  it  should  be  used  also  for  qualitative  research  like  case  study  research  [21]. 

While  the  positivists  still  hold  their  ground,  the  last  couple  of  years  have  seen  an  increased  debate  on  the 
topic  of  scientific  rigour.  It  has  been  argued  that  if  there  is  a  spectrum  of  possible  paradigms,  there  ought 
to  be  a  corresponding  spectrum  of  possible  approaches  to  scientific  rigour  and  research  quality.  Figure  2 
illustrates  such  a  spectrum. 

The  point  of  view  on  scientific  rigour  is  likely  to  depend  on  the  scientific  paradigm.  The  positivist  of  the 
natural  sciences  and  the  objectivist  of  the  social  sciences  will  argue  that  scientific  rigour  must  be 
addressed  with  the  classical  instruments  of  external  validity,  reliability,  construct  validity,  and  internal 
validity.  The  extreme  subjectivist  of  the  social  sciences  will  argue  that  “everything  goes”.  A  more  nuanced 
and  pragmatic  view  is,  perhaps,  that  trustworthiness  should  be  considered  as  an  alternative  to  the  classical 
instruments  when  scientific  rigour  is  discussed  outside  of  the  positivistic  and  objectivistic  paradigms.  The 
pragmatist  will  certainly  argue  that  instruments  for  testing  scientific  rigour  should  be  selected  based  on  the 
context,  i.e.  based  on  the  research  question,  the  research  approach,  and  the  research  methods. 

To  summarise,  there  are  more  scientific  paradigms  than  positivism.  Consequently,  there  are  several 
alternatives  regarding  paradigm,  scientific  method,  scientific  rigour,  and  research  approach. 


4.0  DISCUSSION 

When  the  research  leading  to  this  paper  was  conducted,  there  was  no  consensus  regarding  what  a  military 
concept  is.  Definitions,  terminologies,  taxonomies  and  hierarchies  would  seem  to  differ  even  among  the 
English  speaking  countries,  making  the  situation  even  more  complex  for  countries  with  other  languages, 
especially  for  small  countries  with  an  ambition  to  be  interoperable  with  the  international  CD&E 
community.  As  long  as  e.g.  enabling  concepts  and  applied  concepts  have  different  meanings  to  different 
concept  developers,  international  cooperation  will  remain  unnecessarily  complex.  As  is  further 
accentuated  by  Table  1,  there  is  no  consensus  regarding  concept  terminologies  and  concept  hierarchies  in 
the  English  speaking  countries.  Until  definitions,  terminologies,  taxonomies  and  hierarchies  are 
harmonised  and  standardised,  international  communication  and  cooperation  in  the  area  of  CD&E  will 
continue  to  be  more  complicated  than  it  should  have  to  be.  Smaller  countries  will  also  have  to  decide  with 
whom  they  should  be  interoperable.  Bearing  in  mind  that  the  issues  at  hand,  i.e.  the  military  problems  and 
opportunities  that  are  the  inputs  to  the  process,  are  already  extremely  complex  in  the  first  place,  the  added 
complexity  outlined  above  seem  superfluous. 

Even  if  there  is  no  consensus  regarding  definitions,  terminologies,  taxonomies,  and  hierarchies  in  the 
domain  of  military  concepts,  it  seems  to  be  quite  straightforward  what  the  most  important  elements  of 
military  concept  development  are,  at  least  in  principle.  There  are,  however,  a  few  problems  in  practice 
with  the  definition  in  Chapter  2.0.  Should  all  concept  development  projects  go  all  the  way  through  to  the 
major  experiments?  If  so,  based  on  what  information  should  a  manager  of  an  experimental  platform 
answer  his  superiors  when  they  want  to  put  a  new  development  project  into  the  “concept  development 
factory”?  If  not,  based  on  what  information  should  development  projects  be  terminated  before  reaching  the 
major  experiments?  It  seems  obvious  that  CD&E  must  involve  a  process,  enabling  premature  termination 
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of  concepts,  which  provides  relevant  information  to  decision  makers  during  the  concept  development. 

There  are  also  problems  with  the  suggested  definitions  of  what  CD&E  is.  Is  experimentation  the  only  way 
of  testing  a  developing  concept’s  validity?  Clearly,  as  is  evident  from  the  description  of  validation  and 
IAE,  experimentation  is  not  the  only  form  of  testing.  In  addition  to  tests  in  the  form  of  experiments,  tests 
in  the  form  of  workshops,  seminars,  war  gaming,  Modelling  and  Simulation  (M&S),  etc,  are  suggested  in 
the  literature.  Perhaps  then,  the  name  CD&E  is  misleading?  Perhaps  CD&IAE,  for  Concept  Development 
&  Integrated  Analysis  and  Experimentation  should  be  justified  and  even  more  appropriate?  Or  perhaps  the 
interpretation  of  experimentation  is  too  narrow?  Perhaps  experimentation  should  be  interpreted  so  that  it 
includes  all  the  different  forms  of  testing,  from  workshops  to  full-scale  experiments?  If  so,  however,  most 
detailed  descriptions  concerning  military  experimentation  would  have  to  be  fundamentally  revised,  since 
they  are  based  on  the  “scientific  method”,  i.e.  the  positivist  experimental  method  of  the  natural  sciences, 
which  seems  to  be  an  inadequate  and  inappropriate  method  for  the  evaluation  of  workshops  and  seminars. 

The  lacking  “A”,  for  Analysis,  in  CD&E  is  problematic  from  the  point  of  view  of  a  smaller  country  with 
ambitions  to  use  CD&E.  Perhaps  the  answer  lies  in  fact  that  hitherto,  the  CD&E  method  have  primarily 
been  described  and  outlined  by  large  nations,  like  the  US  or  the  UK,  or  large,  international  organisations 
like  the  NATO  ACT  and  the  TTCP?  Perhaps  these  countries  and  organisations  see  it  as  so  obvious  that 
analysis  should  be  dealt  with  by  an  external  organisation,  so  as  bordering  on  being  not  only  the  self- 
evident  practical  solution,  but  also  being  the  only  theoretical  solution?  Judging  by  their  descriptions  of  the 
CD&E  method,  it  certainly  does  seem  more  than  likely  that  they  presuppose  that  planning,  execution,  and 
evaluation  of  IAE  will  be  handled  by  an  independent  organisation.  This  is  a  luxury  that  smaller  countries 
may,  perhaps,  not  enjoy.  Hence,  from  the  point  of  view  of  a  smaller  country,  where  perhaps  only  one 
organisation,  like  a  JCD&EC,  will  have  to  plan,  execute,  and  evaluate  all  aspects  of  the  testing  of  the 
maturing  concept,  including  all  elements  of  IEA,  perhaps  the  CD&IAE  method,  including  descriptions  of 
all  forms  of  testing  the  concept,  would  be  a  better  name?  Or  perhaps  CD&E  is  too  complex  for  it  to  be 
applicable  without  the  assistance  of  an  external  organisation  for  IAE? 

Military  concept  development  deals  with  highly  complex  problems  and  opportunities.  Hence,  concept 
development  must  be  allowed  to  be  a  very  creative  and  innovative  process,  i.e.  with  a  considerable  portion 
of  “artistic  freedom”.  Even  though  he  does  not  really  want  a  method,  the  concept  developer  must 
eventually  realise  that  concept  development  for  several  reasons  must  follow  some  sort  of  structure.  The 
CD&E  method  must,  however,  be  able  to  provide  enough  analytical  rigour,  while,  at  the  same  time,  it 
avoids  suffocating  creativity  and  innovation. 

From  the  point  of  view  of  the  experimenter,  the  Experiment,  with  a  “capital  E”,  is  his  part  of  CD&E.  To 
satisfy  the  requirements  of  the  experimenter,  the  CD&E  method,  or  CD&E  process,  must  describe  when, 
how,  in  what  form,  etc.,  input  is  to  be  given  from  the  concept  developer  to  the  experimenter.  The  CD&E 
method  does  not  necessarily  have  to  address  this  subject.  A  parallel  description  of  the  CD&E  process  or  a 
Standing  Operating  Procedure  (SOP)  at  the  JCD&EC  will  suffice,  but  experience  demonstrates  beyond 
any  reasonable  doubt  that  this  is  an  essential  aspect  of  CD&E,  and  that  it  must  be  dealt  with  accordingly. 

Regardless  of  source,  the  manager  requires  relevant  information  in  order  so  ensure  “military  relevance”, 
i.e.  that  at  every  given  point  in  time  “the  correct  concepts  are  being  produced”,  and  that  “the  concepts  are 
being  produced  correctly”.  The  manager  definitely  needs  a  formalised  CD&E  process,  dictating  who 
should  produce  which  artefacts,  and  when,  etc.,  so  that  he  can  give  the  HQ  relevant  information  when  it 
requires  it.  This  process  does  not,  however,  have  to  be  a  part  of  the  CD&E  method  per  se. 

The  analyst  has  no  preferences  regarding  the  contents  of  the  ongoing  concept  development.  The  analyst 
does,  however,  make  a  strong  case  for  “analytical  structure”.  Since  military  concept  development  is  a 
complex  issue,  it  seems  justified  that  there  should  be  a  formal,  iterative  method  that  “forces”  concept 
developers  to  study  different  alternative  solutions  and  to  test  the  maturing  concept,  with  increasingly 
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intricate  tests,  as  it  develops.  Quality  assurance  is  an  obvious  and  direct  consequence  of  analytical 
structure.  Another  aspect  of  analytical  structure  is  transparency  of  process.  Since  concept  development 
does  not  results  in  the  “right”  or  the  “wrong”  answer,  it  is  imperative  that  is  possible  to  follow  the  logic  of 
how  the  answer  has  been  produced.  In  combination  with  documented  decisions  regarding  why  specific 
concepts  have  been  developed,  while  others  have  not,  and  documented  decisions  regarding  premature 
termination,  continuation,  approvals,  etc.,  the  transparent  development  process  will  also  contribute  to  the 
establishment  of  an  audit  trail. 

The  scientist  perspective  is  not  one,  but  an  all  but  infinite  number  of  possible  perspectives.  Hitherto,  it  is 
obvious  that  the  positivist  view  of  the  natural  sciences  has  been  allowed  to  assume  a  predominant  position. 
Whether  or  not  this  is  intentional,  based  on  meticulous  deliberations,  is  difficult  to  establish  built  on 
existing  literature  in  the  field.  The  adoption  of  the  positivist  paradigm  of  the  natural  sciences  means  that 
the  research  approach  must  be  deductive,  or  hypothetic-deductive,  resulting  in  the  “scientific  method”,  the 
experimental  method,  or  the  empirical  method.  Within  the  positivistic  paradigm  scientific  rigour  is 
extremely  rigidly  discussed  in  terms  of  external  validity,  reliability,  construct  validity,  and  internal 
validity.  The  selection  of  the  positivist  paradigm  excludes  other  paradigms  like  interpretivism,  scientific 
realism,  and  pragmatism.  Hence,  research  approaches  like  induction  and  abduction  are  also  excluded,  as  is 
different  views  on  scientific  method  and  scientific  rigour. 


5.0  CONCLUSIONS 

There  is  no  consensus  regarding  concept  definitions,  terminologies,  taxonomies  and  hierarchies  within  the 
CD&E  community.  Especially  smaller  countries,  and  countries  with  other  first  languages  than  English, 
would  benefit  tremendously  from  harmonisation  and  standardisation.  Among  other  issues  that  might  be 
resolved,  interoperability  would  certainly  be  facilitated. 

There  is  no  question  that  CD&E  is  an  iterative  concept  development  process,  during  which  a  maturing 
concept’s  validity  is  being  tested  through  increasingly  more  complex  tests,  and  eventually  terminated, 
detained,  or  approved.  From  a  smaller  country  perspective,  however,  the  question  of  validation  activities, 
other  than  large  scale  experiments,  constitutes  a  problem.  Lacking  the  IAE  resources  of  larger  countries, 
enabling  larger  countries  to  rely  on  external  organisations  for  the  planning  and  execution  of  IAE,  smaller 
countries  must  attempt  different  approaches  to  IAE.  One  solution  is  to  describe  IAE  activities  explicitly  in 
the  CD&E  method,  perhaps  motivating  a  change  of  name  to  the  CD&IAE  method.  Another  solution  is  to 
accept  that  smaller  countries  can  not  utilise  the  CD&E  method  as  it  is  described  by  larger  countries. 

There  is  no  “A”  in  CD&E,  neither  for  “Analysis”  nor  for  “Anarchy”.  The  CD&E  method  must  certainly 
respect  the  requirement  for  a  necessary  degree  of  “Artistic  Freedom”  for  the  concept  developers,  but  it 
definitely  can  not  allow  anarchy.  Quite  to  the  contrary,  the  CD&E  method  must  instead  provide  enough 
“Analytical  Structure”  in  order  to  assure  concept  quality  and  concept  development  quality,  ensure 
transparency  of  process,  and  enable  an  audit  trail.  The  CD&E  method  must  also  assist  the  production  of 
relevant  information  to  the  decision  makers,  so  that  “Military  Relevance”  can  be  guaranteed,  i.e.  that  at 
every  given  point  in  time  “the  correct  concepts  are  being  produced”,  and  that  “the  concepts  are  being 
produced  correctly”.  The  issue  of  “Scientific  Rigour”  in  CD&E  would  seem  to  require  further  deliberation 
within  the  CD&E  community.  It  is  far  from  clear  whether  or  not  the  positivistic  paradigm  has  been 
selected  intentionally.  Regardless  of  which,  the  CD&E  community  ought  to  deliberately  consider,  perhaps 
reconsider,  which  paradigm  that  should  be  used  for  CD&E.  It  is  far  from  obvious  that  the  positivistic 
paradigm  of  the  natural  sciences  is  the  most  appropriate  paradigm  for  CD&E.  Conversely,  it  seems  likely 
that  paradigms  from  the  social  sciences,  such  as  interpretivism,  scientific  realism,  or  pragmatism,  should 
be  better  suited  to  address  CD&E.  With  such  paradigms,  the  issues  of  scientific  method  and  scientific 
rigour  would  come  into  an  entirely  new  light. 
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6.0  ACRONYMS  AND  ABBREVIATIONS 


ACT: 

AF: 

C2: 

C2-system  M: 
C2-system  O: 
C2-system  P: 
C2-system  T: 
CA: 

CD&E: 

CD&IAE: 

CDP: 

CFEC: 

CONEMP 

CONOPS: 

CONUSE: 

DBA: 

DP  A: 

DRA: 

EBAO: 

EBO: 

FMV: 

FOE 

HQ: 

IAE: 

IAEC: 

IBM: 

ICT: 

JCD&EC: 

JFC: 

LL: 

LOE: 

M&S: 

ME: 

MIE: 


Allied  Command  Transformation 
Armed  Forces 
Command  and  Control 

Swedish  transformational  development  project  for  C2  system  methodology 

Swedish  transformational  development  project  for  C2  system  organisation 

Swedish  transformational  development  project  for  C2  system  personnel 

Swedish  transformational  development  project  for  C2  system  technology 

Comprehensive  Approach 

Concept  Development  and  Experimentation 

Concept  Development  &  Integrated  Analysis  and  Experimentation 

Capability  Development  Process 

Canadian  Forces  Experimentation  Centre 

Concept  of  Employment: 

Concept  of  Operations 
Concept  of Use 

Dominant  Battle-space  Awareness 

Defence  Procurement  Agency 

Defence  Research  Agency 

Effects  Based  Approach  to  Operations 

Effects  Based  Operations 

The  Swedish  Defence  Materiel  Administration 

The  Swedish  Defence  Research  Agency 

Head  Quarters 

Integrated  Analysis  and  Experimentation 

Integrated  Analysis  and  Experimentation  Campaign 

International  Business  Machines 

Information  and  Communication  Technology 

Joint  Concept  Development  and  Experimentation  Centre 

Joint  Forces  Command 

Lessons  Learned 

Limited  Objective  Experiment 

Modelling  and  Simulation 

Main  Event 

Minor  Integrating  Event 
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M.O.:  Modus  Operandi 

MoD:  Ministry  of  Defence 

NATO:  North  Atlantic  Treaty  Organisation 

NBD:  Network-Based  Defence 

NOW:  Network  Centric  Warfare 

OR:  Operational  Research 

PE:  Precision  Engagement 

RMA:  Revolution  in  Military  Affairs 

RUP:  Rational  Unified  Process 

SE:  Systems  Engineering 

SNDC:  Swedish  National  Defence  College 

SOP:  Standing  Operating  Procedure 

SU:  Soviet  Union 

TTCP:  The  Technical  Cooperation  Program  (Australia,  New  Zealand,  US,  UK  and  Canada) 

UK:  United  Kingdom  (of  Great  Britain  and  Northern  Ireland) 

US:  United  States  (of  America) 

WFE:  Warfighting  Experimentation 

WP:  Warsaw  Pact 
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